fix(test): make the sweep's flaky e2e and packed suites deterministic - #435
Conversation
mcp-page-app-browser (8-of-7 closes): the MCP App frame relay posted its
teardown into a proxy document that had not signaled readiness, so nothing
could acknowledge it and the 5 s force-close timer added a second close
record whenever the test outlived the budget. The relay now force-closes
immediately when the proxy never reported ready (no window can answer), and
the test waits on each binding's initialized/closed lifecycle records and
the sandbox server's own request event instead of DOM presence.
mcp-app-real: the Run -> /api/runtime/apps wait now sequences run admission,
run settlement, and the create request, failing fast with run diagnostics;
the second /close wait first waits for the reopened app to initialize.
dev-workbench (double config load): the fixture wrote its config-call log
and provider context into the watched project root, so every load was a
source change that rebuilt and re-loaded the config. Both records now live
outside the watched tree (provider storageRoot; assets root).
packed-release (epoch mismatch): source edits now wait for the packed dev
server's own watcher rebuild via project_status attempt ids before the
Overview Rebuild click, so one edit is one build and no late epoch replaces
the one later phases pinned.
public-api-packed (30 s timeouts): npm caches are now per worker rather
than per command, and the file warms the cache once in a budgeted beforeAll,
so no per-test budget spans a cold ~180 MB registry download.
Rspack persistent cache ("Transaction already in progress"): Rslib enables
the persistent cache by default and keys it by config root, so two test
workers rebuilding packages/agent-bundle into isolated dists shared one lock.
Spawned builds now get a per-command cache directory via
AGENT_BUNDLE_RSLIB_CACHE_DIRECTORY.
docs/local-ci.md records the retry policy for the registry-side
EATTESTATIONVERIFY failure of `npm audit signatures`.
🦋 Changeset detectedLatest commit: 308b245 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
commit: |
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 participant
Summary
Deterministic fixes for the e2e/packed suites reported flaky across lanes and CI. Every fix is at the source of the race (event-ordered waits, per-worker isolation, a product fix in the MCP App frame relay); no retries, no wall-clock padding, no raised timeouts.
mcp-page-app-browserclose count (expected 8 to be 7,expected [] to equal ['/'])McpAppFrameRelay.close()postedui/resource-teardowninto a proxy iframe that had not signaled readiness (still loading /about:blank, whose origin never matchestargetOrigin), so nothing could acknowledge it; the 5 s force-close timer then added a second close record whenever the test outlived the budget (the failing CI run took 6.97 s vs ~1.5 s). The sandbox-request assertion read the server log right after the iframe entered the DOM, before its document request reached the server.closingfor the whole budget (packages/workbench/src/mcp/mcp-app-frame.tsx, unit test proves the 30 s budget is never armed). Test: wait on each binding'sinitialized/ close route records and on the sandbox server's own request event; assert the exact per-binding close sequence.mcp-app-realwaitForRequest(Run →POST /api/runtime/apps)/closewait now waits for the reopened app'sui/notifications/initializedfirst (a preview closed before its proxy loads is released by DELETE, by design after the relay fix).dev-workbench"double config load"config-calls.ndjsonand wroteprovider-context.jsoninto the watched project root. The dev watcher treats every non-ignored path as source (the project snapshot is broad by design), so each write was a source change → rebuild →prepare('dev')→ config re-load → append → another rebuild: a self-sustaining loop that reached 12 loads in 2 s once the read was delayed.storageRoot(.agent-bundle/runtime/<session>), config-call log beside the workbench assets. Assertion unchanged (exactly onedevload). Source behaves per its documented contract; no src change.dev-workbench-packagingpnpm buildfrom inside the unit pool collided with another worker's build (Failed to generate declaration files). Since moved to the prebuilt packed pool by earlier PRs. Remaining hazard: itsrslib build --dist-path <isolated>shares Rslib's persistent cache withpacked-consumer's build (see Rspack item).installedEnvironment()on the spawned build → per-command cache directory.packed-release.e2e"epoch mismatch" (expected 'a9dd…' to be 'd38a…'in phaseinvalid edit retains stale epoch B)replaceWatchedSourcewith an immediate Overview Rebuild click, racing the watcher's own rebuild of the same write; whichever build landed last silently replaced the epoch later phases had pinned (the patternreplaceWatchedSourceAndAwaitRebuildalready fixed for in-process servers).replaceSourceAndAwaitWatcherRebuild: captureproject_statusattempt ids, replace, wait for the first completed attempt not known before the write, then click Rebuild. The two-file epoch C edit is sequenced.public-api-packed30 s timeoutsisolatedCommandEnvironment()gave every spawned command a fresh npm cache, so everynpm install <tarball>was cold: ~180 MB of registry tarballs per test (8.9 s cold vs 4.3 s offline here; a hosted runner's npm cache is empty at job start —pnpm/setupcaches only the pnpm store — so Release gates and Verify pay it for every install).rstestWorkerNpmCacheDirectory), and the file warms it once in abeforeAllwith its own 180 s budget (justified in the comment), so no per-test 30 s budget spans a cold download.--prefer-offlinethen serves everything from disk.Transaction already in progresscreateConstantRsbuildConfigsetsperformance.buildCache: true; Rsbuild keys the directory as<config root>/node_modules/.cache/rspack, never by--dist-path.packed-consumeranddev-workbench-packagingboth spawnrslib build --config packages/agent-bundle/rslib.config.tsfrom parallel packed-pool workers → one lock. Reproduced locally: 1 of 13 cold concurrent pairs failed withTransaction already in progress by process 2892811:node in directory '…/packages/agent-bundle/node_modules/.cache/rspack/esm-production/rspack_v_ef3dc523a56e4030/.temp'; 0 of 8 with isolated directories.packages/agent-bundle/rslib.config.tshonoursAGENT_BUNDLE_RSLIB_CACHE_DIRECTORY;isolatedCommandEnvironment()sets it per command.pnpm buildkeeps the default warm cache.EATTESTATIONVERIFY(npm audit signatures)@modelcontextprotocol/server@2.0.0 failed to verify attestation: Unexpected end of JSON input).docs/local-ci.md; no code change.Evidence
ci.ymlruns (2026-08-28 → 2026-09-03) plus 3release.ymlfailures. Cited runs: 33734384098 (dev-workbench double load + 8-of-7 closes), 33735363205 ([]vs['/']), 33420032167 (waitForRequest30 s at the Run step), 33154508245 (packed-release epoch mismatch), 33232848276 (dev-workbench-packaging dts collision), 33584654855 (EATTESTATIONVERIFY).devconfig loads (after: exactly 1 with the same delay); new relay unit test on the old source → hangs on the 30 s timer (test timed out in 30000ms); concurrent rslib builds on a shared cold cache → 1/13Transaction already in progress.AGENT_BUNDLE_TEST_TIME_SCALE=4): dev-workbench runtime-once 10/10, mcp-page-app-browser 5/5 (+12/12 under 4-way concurrent stress and 3 CPU-throttled runs earlier), mcp-app-real 3/3, packed-release 3/3, public-api-packed + dev-workbench-packaging + packed-consumer 3/3 pool runs.Test plan
pnpm typecheckpnpm lintpnpm test:unit— 3002 passed, 1 failed:native-claude-contract.test.ts › fails closed when the candidate plugin, exact Skill event, or subscription auth source is absent(test timed out in 5000ms). Pre-existing: fails identically on a pristineorigin/maincheckout in this environment; untouched by this PR.pnpm build && pnpm test:integration:runpnpm test:packed(touched files) — 8/8Review status
308b245e8(Verify Node 24, Release gates, Host install proofs, Examples check, RSC runtime micro-eval, Docs site, Changeset present, Dependency review).chatgpt-codex-connector: three requests (automatic on open,@codex reviewat 18:50 UTC and 19:07 UTC) all answered "You have reached your Codex usage limits for code reviews"; zero review threads.133361eab(the fix),308b245e8(changeset PR reference). Merged on green CI per the lane policy; any later codex threads will be answered in a follow-up PR.